{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BERT文本分类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "😋😋公众号算法美食屋后台回复关键词：**torchkeras**，获取本文notebook源代码和数据集下载链接。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本范例我们微调transformers中的BERT来处理文本情感分类任务。\n",
    "\n",
    "我们的数据集是美团外卖的用户评论数据集。\n",
    "\n",
    "模型目标是把评论分成好评(标签为1)和差评(标签为0)。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#安装库\n",
    "#!pip install datasets \n",
    "#!pip install transformers[torch]\n",
    "#!pip install torchkeras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一，准备数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "准备数据阶段主要需要用到的是datasets.Dataset 和transformers.AutoTokenizer。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1，数据加载"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HuggingFace的datasets库提供了类似TensorFlow中的tf.data.Dataset的功能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import pandas as pd \n",
    "\n",
    "import torch \n",
    "from torch.utils.data import DataLoader \n",
    "\n",
    "import datasets \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"data/waimai_10k.csv\")\n",
    "ds = datasets.Dataset.from_pandas(df) \n",
    "ds = ds.shuffle(42) #打乱顺序\n",
    "ds = ds.rename_columns({\"review\":\"text\",\"label\":\"labels\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'labels': 0, 'text': '晚了半小时，七元套餐饮料就给的罐装的可乐，真是可以'}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['晚了半小时，七元套餐饮料就给的罐装的可乐，真是可以',\n",
       " '很好喝！天天都喝～～',\n",
       " '东西很少，像半分每次都是这样失望',\n",
       " '配送比较慢（不是高峰时间点的结果1个多小时才送到）；菜品备注了“老人吃请少油少盐”，结果还是很咸很油，哎…失望']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds[0:4][\"text\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2，文本分词"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "transformers库使用tokenizer进行文本分词。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BertTokenizerFast(name_or_path='bert-base-chinese', vocab_size=21128, model_max_length=512, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'unk_token': '[UNK]', 'sep_token': '[SEP]', 'pad_token': '[PAD]', 'cls_token': '[CLS]', 'mask_token': '[MASK]'}, clean_up_tokenization_spaces=True)\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer #BertTokenizer\n",
    "tokenizer = AutoTokenizer.from_pretrained('bert-base-chinese') #需要和模型一致\n",
    "print(tokenizer) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'input_ids': [101, 3241, 749, 1288, 2207, 3198, 8024, 673, 1039, 1947, 7623, 7650, 3160, 2218, 5314, 4638, 5380, 6163, 4638, 1377, 727, 8024, 4696, 3221, 1377, 809, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}\n"
     ]
    }
   ],
   "source": [
    "#tokenizer可以使用 __call__,encode,encode_plus,batch_encode_plus等方法编码\n",
    "#可以使用decode,batch_decode等方法进行解码\n",
    "text_codes = tokenizer(text = '晚了半小时，七元套餐饮料就给的罐装的可乐，真是可以',\n",
    "                       text_pair = None,\n",
    "                       max_length = 100, #为空则默认为模型最大长度，如BERT是512,GPT是1024\n",
    "                       truncation = True,\n",
    "                       padding= 'do_not_pad') #可选'longest','max_length','do_not_pad'\n",
    "\n",
    "#input_ids是编码后的数字，token_type_ids表示来自第1个句子还是第2个句子\n",
    "#attention_mask在padding的位置是0其它位置是1\n",
    "print(text_codes) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[CLS]'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.decode(text_codes[\"input_ids\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['[CLS]',\n",
       " '晚',\n",
       " '了',\n",
       " '半',\n",
       " '小',\n",
       " '时',\n",
       " '，',\n",
       " '七',\n",
       " '元',\n",
       " '套',\n",
       " '餐',\n",
       " '饮',\n",
       " '料',\n",
       " '就',\n",
       " '给',\n",
       " '的',\n",
       " '罐',\n",
       " '装',\n",
       " '的',\n",
       " '可',\n",
       " '乐',\n",
       " '，',\n",
       " '真',\n",
       " '是',\n",
       " '可',\n",
       " '以',\n",
       " '[SEP]']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.batch_decode(text_codes[\"input_ids\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tokens= ['晚', '了', '半', '小', '时', '，', '七', '元', '套', '餐', '饮', '料', '就', '给', '的', '罐', '装', '的', '可', '乐', '，', '真', '是', '可', '以']\n",
      "ids =  [3241, 749, 1288, 2207, 3198, 8024, 673, 1039, 1947, 7623, 7650, 3160, 2218, 5314, 4638, 5380, 6163, 4638, 1377, 727, 8024, 4696, 3221, 1377, 809]\n"
     ]
    }
   ],
   "source": [
    "tokens = tokenizer.tokenize(ds['text'][0])\n",
    "print(\"tokens=\",tokens)\n",
    "ids = tokenizer.convert_tokens_to_ids(tokens)\n",
    "print(\"ids = \",ids)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3，传入DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map (num_proc=2):   0%|          | 0/11987 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds_encoded = ds.map(lambda example:tokenizer(example[\"text\"],\n",
    "                    max_length=50,truncation=True,padding='max_length'),\n",
    "                    batched=True,\n",
    "                    batch_size=20,\n",
    "                    num_proc=2) #支持批处理和多进程map \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'labels': tensor(0),\n",
       " 'input_ids': tensor([ 101, 3241,  749, 1288, 2207, 3198, 8024,  673, 1039, 1947, 7623, 7650,\n",
       "         3160, 2218, 5314, 4638, 5380, 6163, 4638, 1377,  727, 8024, 4696, 3221,\n",
       "         1377,  809,  102,    0,    0,    0,    0,    0,    0,    0,    0,    0,\n",
       "            0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,\n",
       "            0,    0]),\n",
       " 'token_type_ids': tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0]),\n",
       " 'attention_mask': tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "         1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0])}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#转换成pytorch中的tensor \n",
    "ds_encoded.set_format(type=\"torch\",columns = [\"input_ids\",'attention_mask','token_type_ids','labels'])\n",
    "#ds_encoded.reset_format() \n",
    "ds_encoded[0]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#分割成训练集和测试集\n",
    "ds_train_val,ds_test = ds_encoded.train_test_split(test_size=0.2).values()\n",
    "ds_train,ds_val = ds_train_val.train_test_split(test_size=0.2).values() \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#在collate_fn中可以做动态批处理(dynamic batching)\n",
    "\n",
    "def collate_fn(examples):\n",
    "    return tokenizer.pad(examples) #return_tensors='pt'\n",
    "\n",
    "#以下方式等价\n",
    "#from transformers import DataCollatorWithPadding\n",
    "#collate_fn = DataCollatorWithPadding(tokenizer=tokenizer)\n",
    "\n",
    "dl_train = torch.utils.data.DataLoader(ds_train, batch_size=16, collate_fn = collate_fn)\n",
    "dl_val = torch.utils.data.DataLoader(ds_val, batch_size=16,  collate_fn = collate_fn)\n",
    "dl_test = torch.utils.data.DataLoader(ds_test, batch_size=16,  collate_fn = collate_fn)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You're using a BertTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
     ]
    }
   ],
   "source": [
    "for batch in dl_train:\n",
    "    break "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二，定义模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个完整的模型(Model)包括模型架构(Architecture)和模型权重(Checkpoints/Weights)。\n",
    "\n",
    "transformers提供了3种指定模型架构的方法。\n",
    "\n",
    "* 第1种是指定模型架构(如: from transformers import BertModel)\n",
    "\n",
    "* 第2种是自动推断模型架构(如: from transformers import AutoModel)\n",
    "\n",
    "* 第3种是自动推断模型架构并自动添加Head (如: from transformers import AutoModelForSequenceClassification )\n",
    "\n",
    "第1种方案和第2种方案用户可以灵活地根据自己要做的任务设计Head，并且需要对基础模型有一定的了解。\n",
    "\n",
    "\n",
    "此处我们使用第3种方案。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at bert-base-chinese were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight']\n",
      "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-chinese and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "dict_keys(['bert', 'dropout', 'classifier'])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import AutoModelForSequenceClassification \n",
    "\n",
    "#加载模型 (会添加针对特定任务类型的Head)\n",
    "model = AutoModelForSequenceClassification.from_pretrained('bert-base-chinese',num_labels=2)\n",
    "dict(model.named_children()).keys() \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以用一个batch的数据去试算一下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "output = model(**batch)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.6762, grad_fn=<NllLossBackward0>)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output.loss "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三，训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面使用我们的梦中情炉 torchkeras 来实现最优雅的微调训练循环。🤗🤗\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchkeras import KerasModel \n",
    "\n",
    "#我们需要修改StepRunner以适应transformers的数据集格式\n",
    "\n",
    "class StepRunner:\n",
    "    def __init__(self, net, loss_fn, accelerator, stage = \"train\", metrics_dict = None, \n",
    "                 optimizer = None, lr_scheduler = None\n",
    "                 ):\n",
    "        self.net,self.loss_fn,self.metrics_dict,self.stage = net,loss_fn,metrics_dict,stage\n",
    "        self.optimizer,self.lr_scheduler = optimizer,lr_scheduler\n",
    "        self.accelerator = accelerator\n",
    "        if self.stage=='train':\n",
    "            self.net.train() \n",
    "        else:\n",
    "            self.net.eval()\n",
    "    \n",
    "    def __call__(self, batch):\n",
    "        \n",
    "        out = self.net(**batch)\n",
    "        \n",
    "        #loss\n",
    "        loss= out.loss\n",
    "        \n",
    "        #preds\n",
    "        preds =(out.logits).argmax(axis=1) \n",
    "    \n",
    "        #backward()\n",
    "        if self.optimizer is not None and self.stage==\"train\":\n",
    "            self.accelerator.backward(loss)\n",
    "            self.optimizer.step()\n",
    "            if self.lr_scheduler is not None:\n",
    "                self.lr_scheduler.step()\n",
    "            self.optimizer.zero_grad()\n",
    "        \n",
    "        all_loss = self.accelerator.gather(loss).sum()\n",
    "        \n",
    "        labels = batch['labels']\n",
    "        acc = (preds==labels).sum()/((labels>-1).sum())\n",
    "        \n",
    "        all_acc = self.accelerator.gather(acc).mean()\n",
    "        \n",
    "        #losses\n",
    "        step_losses = {self.stage+\"_loss\":all_loss.item(), self.stage+'_acc':all_acc.item()}\n",
    "        \n",
    "        #metrics\n",
    "        step_metrics = {}\n",
    "        \n",
    "        if self.stage==\"train\":\n",
    "            if self.optimizer is not None:\n",
    "                step_metrics['lr'] = self.optimizer.state_dict()['param_groups'][0]['lr']\n",
    "            else:\n",
    "                step_metrics['lr'] = 0.0\n",
    "        return step_losses,step_metrics\n",
    "    \n",
    "KerasModel.StepRunner = StepRunner \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.AdamW(model.parameters(), lr=3e-5)\n",
    "\n",
    "keras_model = KerasModel(model,\n",
    "                   loss_fn=None,\n",
    "                   optimizer = optimizer\n",
    "                   )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< ⚡️ cuda is used >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiEAAAGJCAYAAABcsOOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABkBklEQVR4nO3deXiM19sH8O9kSDIkYs9OIvYIQUhRVKmUova1tbW0tdRS2tjVFrRNY6utioq1BG2plhRFbUX8KGInglhKQsg2c94/zpuJSSb78swk3891PdfMnDnzzD0j8tw5q0oIIUBERERUwCyUDoCIiIiKJiYhREREpAgmIURERKQIJiFERESkCCYhREREpAgmIURERKQIJiFERESkCCYhREREpAgmIURERKQIJiFEeWT69OlQqVR49OiR0qEUmJs3b0KlUmHNmjVKh0JEZohJCJGZmzNnDnbs2KF0GEVefHw8vvjiCzg5OUGj0cDX1xd79+7N8us3bdqEBg0awNraGhUqVMAHH3yQJqGNiIjAl19+icaNG6NMmTIoX7483njjDezbty/N+UJDQzF48GBUr14dJUqUQJUqVfDhhx/i3r17aerqdDosW7YM3t7esLGxgb29Pdq1a4e///47+18EUTYwCSEyc0xCTMPAgQMRGBiIfv36YcGCBVCr1Wjfvj0OHz6c6WuXLl2KPn36oGzZsggMDMSQIUOwadMmtG7dGnFxcfp6O3fuxLx581C1alXMmjULU6ZMwbNnz/DWW29h9erVBuf84osvcODAAXTp0gULFy5E7969sWXLFtSvXx/37983qDt+/Hh88skn8PLyQmBgID777DNcvnwZLVu2xIkTJ/LmCyIyRhBRnpg2bZoAIB4+fFig71uyZEkxYMCAAn3PZDdu3BAAxOrVqxV5f1Nx/PhxAUB89dVX+rKXL18KDw8P0aRJkwxfGx8fL0qXLi1atGghdDqdvvyXX34RAMTChQv1ZefPn0/z8xUXFydq1qwpXFxcDMoPHjwotFptmjIAYtKkSfqyxMREodFoRPfu3Q3qXr9+XQAQn376aSafnijn2BJClMcePXqEnj17olSpUihXrhxGjRpl8NdssuDgYDRs2BAajQZly5ZF7969ERERYVDnypUr6NatGxwcHGBtbQ0XFxf07t0b0dHRAACVSoXY2FisXbsWKpUKKpUKAwcONBpXVFQUihUrhi+//DLNc+Hh4VCpVFi8eDEA4L///sO4cePg5eUFGxsblCpVCu3atcPZs2dz+e0ACQkJmDp1Kho2bAg7OzuULFkSzZs3x/79+9PU1el0WLBgAby8vPTdFG+//Tb++ecfg3rBwcFo3LgxSpQogTJlyqBFixb4448/ch1rVm3duhVqtRpDhw7Vl1lbW+ODDz7A0aNH0/y7vur8+fN4+vQpevXqBZVKpS/v0KEDbGxssGnTJn2Zp6cnypcvb/B6KysrtG/fHnfu3MGzZ8/05S1atICFheGv+BYtWqBs2bK4ePGiviwxMREvX76Evb29Qd2KFSvCwsICGo0mi98CUfYVUzoAosKmZ8+ecHNzQ0BAAI4dO4aFCxfiyZMn+PHHH/V1Zs+ejSlTpqBnz5748MMP8fDhQyxatAgtWrTAmTNnULp0aSQkJMDPzw/x8fEYOXIkHBwcEBkZiV9//RVPnz6FnZ0d1q1bhw8//BCNGzfWXwA9PDyMxmVvb4+WLVtiy5YtmDZtmsFzmzdvhlqtRo8ePQAA169fx44dO9CjRw+4u7sjKioKy5cvR8uWLXHhwgU4OTnl+PuJiYnB999/jz59+mDIkCF49uwZVq1aBT8/P5w4cQLe3t76uh988AHWrFmDdu3a4cMPP0RSUhIOHTqEY8eOwcfHBwDw5ZdfYvr06WjatClmzJgBS0tLHD9+HH/++Sfatm2bbhzx8fEGF+2MpL7wp3bmzBlUr14dpUqVMihv3LgxACAsLAyurq7pxgHA6MVeo9HgzJkz0Ol0aRKKV92/fx8lSpRAiRIlMozz+fPneP78ucHnSR6/smbNGjRp0gTNmzfH06dPMXPmTJQpU8YgsSLKc0o3xRAVFsndMZ06dTIoHzZsmAAgzp49K4QQ4ubNm0KtVovZs2cb1Dt37pwoVqyYvvzMmTMCgPjpp58yfN/sdMcsX75cABDnzp0zKK9du7Z488039Y/j4uLSNOXfuHFDWFlZiRkzZhiUIZvdMUlJSSI+Pt6g7MmTJ8Le3l4MHjxYX/bnn3+m2x2Q3G1x5coVYWFhIbp06ZIm3le7NoxZvXq1AJClIzOenp4G31+yf//9VwAQy5YtS/e1Dx8+FCqVSnzwwQcG5ZcuXdK//6NHj9J9/ZUrV4S1tbV4//33M41z5syZAoAIDQ1Nc44GDRoYfOYqVaqIS5cuZXpOotxgSwhRHhs+fLjB45EjR+K7777D7t27UbduXYSEhECn06Fnz54Gsx8cHBxQrVo17N+/HxMnToSdnR0A4Pfff0f79u0z/Ss3K7p27Yrhw4dj8+bNqFOnDgDZHXDhwgWMGjVKX8/Kykp/X6vV4unTp7CxsUGNGjVw+vTpXMWgVquhVqsByO6Wp0+fQqfTwcfHx+Dc27Ztg0qlStNqA0DfbbFjxw7odDpMnTo1TUvBq10bxvj5+WVr9kpGXr58afCdJbO2ttY/n57y5cujZ8+eWLt2LWrVqoUuXbogMjISI0eORPHixfXdJca8ePECPXr0gEajwdy5czOM8a+//sKXX36Jnj174s033zR4ztbWFp6enmjSpAlat26N+/fvY+7cuejcuTMOHTqUaUsQUY4pnQURFRbJLSHXr183KE9ISBAWFhbio48+EkII8cknn2T4V3fdunX1rx07dqwAIDQajWjbtq1YvHixePr0qcH5szsw1c/PT1SvXl3/ePLkyaJYsWIGAx61Wq0IDAwUVatWFWq12iC+Vq1a6evldGDqmjVrhJeXlyhevLjBud3d3fV13n77beHs7JzheT7++GNhYWGRpmWloOWmJUQIIZ4+fSo6depk8F289957omvXrgKAePLkSZrXJCUliY4dOwpLS8s0LRupXbx4UZQtW1Z4e3uLmJgYg+cSExNFnTp1xIgRIwzKL1++LIoXLy4+//zzDM9NlBtsCSHKZ6n/ItfpdFCpVPjtt9/0LQKvsrGx0d//5ptvMHDgQOzcuRN//PEHPv30U/1YExcXlxzF07t3bwwaNAhhYWHw9vbGli1b0Lp1a4O/dufMmYMpU6Zg8ODBmDlzJsqWLQsLCwuMHj0aOp0uR++bLDg4GAMHDkTnzp0xfvx4VKxYEWq1GgEBAbh27Vquzp0dL1++1A/wzYyDg0OGzzs6OiIyMjJNefKaHJmNobGzs8POnTtx+/Zt3Lx5E5UrV0blypXRtGlTVKhQAaVLl07zmiFDhuDXX3/F+vXr07RsvCoiIgJt27aFnZ0ddu/eDVtbW4Pn//rrL5w/fx6BgYEG5dWqVUOtWrVw5MiRDGMnyg0mIUR57MqVK3B3d9c/vnr1KnQ6Hdzc3ADIgaNCCLi7u6N69eqZns/LywteXl6YPHky/v77bzRr1gzLli3DrFmzAGTe7ZBa586d8dFHH2Hz5s0AgMuXL2PChAkGdbZu3YpWrVph1apVBuVPnz7NddP81q1bUaVKFYSEhBjEnrrbxcPDA7///jv+++8/lC1b1ui5PDw8oNPpcOHCBYMBrVmxefNmDBo0KEt1hRAZPu/t7Y39+/cjJibGYHDq8ePH9c9nRaVKlVCpUiUA8rs+deoUunXrlqbe+PHjsXr1agQFBaFPnz7pnu/x48do27Yt4uPjERoaCkdHxzR1oqKiAMhut9QSExORlJSUpdiJcoJTdIny2JIlSwweL1q0CADQrl07AHJchlqtxpdffpnm4iaEwOPHjwHIWSSpLwBeXl6wsLDQz6gAgJIlS+Lp06dZjq906dLw8/PDli1bsGnTJlhaWqJz584GddRqdZrYfvrpJ6N/7WdXcuvPq+c/fvw4jh49alCvW7duEEIYnVKc/NrOnTvDwsICM2bMSNNCk1nikDwmJCtHZrp37w6tVosVK1boy+Lj47F69Wr4+voazIy5ffs2Ll26lOk5J0yYgKSkJIwZM8ag/KuvvsLXX3+NiRMnGozjSS02Nhbt27dHZGQkdu/ejWrVqhmtl5wIvzoVGABOnz6N8PBw1K9fP9NYiXKKLSFEeezGjRvo1KkT3n77bRw9ehTBwcHo27cv6tWrB0D+9T5r1ixMmDABN2/eROfOnWFra4sbN25g+/btGDp0KMaNG4c///wTI0aMQI8ePVC9enUkJSVh3bp1UKvVBn8dN2zYEPv27UNgYCCcnJzg7u4OX1/fDGPs1asX3nvvPXz33Xfw8/NL09zfoUMHzJgxA4MGDULTpk1x7tw5rF+/HlWqVMn199OhQweEhISgS5cueOedd3Djxg0sW7YMtWvXxvPnz/X1WrVqhffffx8LFy7ElStX8Pbbb0On0+HQoUNo1aoVRowYgapVq2LSpEmYOXMmmjdvjq5du8LKygonT56Ek5MTAgIC0o3D0dHRaMtATvj6+qJHjx6YMGECHjx4gKpVq2Lt2rW4efNmmtak/v374+DBgwZJ0ty5c3H+/Hn4+vqiWLFi2LFjB/744w/MmjULjRo10tfbvn07Pv/8c31XSXBwsMG533rrLf16H/369cOJEycwePBgXLx40WBtEBsbG33i2bBhQ7z11ltYu3YtYmJi0LZtW9y7dw+LFi2CRqPB6NGj8+Q7IjJKqcEoRIVN8sDUCxcuiO7duwtbW1tRpkwZMWLECPHy5cs09bdt2yZef/11UbJkSVGyZElRs2ZNMXz4cBEeHi6EkCtWDh48WHh4eAhra2tRtmxZ0apVK7Fv3z6D81y6dEm0aNFCaDQaASBLg1RjYmL09YODg9M8HxcXJz777DPh6OgoNBqNaNasmTh69Kho2bKlaNmypb5eTgam6nQ6MWfOHFG5cmVhZWUl6tevL3799VcxYMAAUblyZYO6SUlJ4quvvhI1a9YUlpaWokKFCqJdu3bi1KlTBvV++OEHUb9+fWFlZSXKlCkjWrZsKfbu3ZvlmPLCy5cvxbhx44SDg4OwsrISjRo1Env27ElTr2XLlmmm/f7666+icePGwtbWVpQoUUK89tprYsuWLWlem/wzlt6xf/9+fd3KlSunWy/19/zixQsxY8YMUbt2baHRaISdnZ3o0KGDOHPmTF58NUTpUgmRSZslERERUT7gmBAiIiJSBMeEEFGeSEhIwH///ZdhHTs7O+5FQkR6TEKIKE/8/fffaNWqVYZ1Vq9ene4Ge0RU9HBMCBHliSdPnuDUqVMZ1vH09MyzGSlEZP6YhBAREZEiODCViIiIFMExIUbodDrcvXsXtra22V4Sm4iIqCgTQuDZs2dwcnJKs7t1akxCjLh7967BMstERESUPREREZlutMkkxIjkXSYjIiIMNqMiIiKijMXExMDV1TXNjs3GMAkxIrkLplSpUkxCiIiIciArwxk4MJWIiIgUwSSEiIiIFMEkhIiIiBTBMSE5JIRAUlIStFqt0qFQNhUvXhxqtVrpMIiIijwmITmQkJCAe/fu4cWLF0qHQjmgUqng4uICGxsbpUMhIirSmIRkk06nw40bN6BWq+Hk5ARLS0suaGZGhBB4+PAh7ty5g2rVqrFFhIhIQUxCsikhIQE6nQ6urq4oUaKE0uFQDlSoUAE3b95EYmIikxAiynNaLXDoEHDvHuDoCDRvDvBXjXFMQnIos6VoyXSx5YqI8ktICDBqFHDnTkqZiwuwYAHQtatycZkqXkmJiIjyQEgI0L27YQICAJGRsjwkRJm4TBmTECIiolzSamULiBBpn0suGz1a1qMUTEIUpNUCBw4AGzfKW3P64XRzc0NQUJDSYRARKeLJE+DGjZTHBw6kbQF5lRBARIQcKwIAK1YA69cDf/0lzxMfn6/hGmUK1yCOCVGIEv2Gb7zxBry9vfMkeTh58iRKliyZ+6CIiEzYtWvApUvyCA9PuX3wQA44/esvWe/Bg6yd7949mZCMHQvExho+Z28vrwNvvAF8/XVK+cmTQPnygLMzYGmZJx/LZMauMAlRQHK/Yepmu+R+w61blRnAJISAVqtFsWKZ/1hUqFChACIiIpLyc8bJ06cysQgPBxISgA8/THmuZUv5u9mYZ89S7js6Zu29HB2BxET5Oz4iQiYBERGyJSQqSh5OTin1hQBatUpJWOztAVdXmTC4ugKNGgHvv59SPykJyOxXuEldgwSlER0dLQCI6OjoNM+9fPlSXLhwQbx8+TLNc8+fp38kV09KEsLFRQj5z5/2UKnk80lJGZ83uwYMGCAAGByrV68WAMTu3btFgwYNRPHixcX+/fvF1atXRadOnUTFihVFyZIlhY+Pj9i7d6/B+SpXriy+/fZb/WMAYuXKlaJz585Co9GIqlWrip07d2YptqSkJDF48GDh5uYmrK2tRfXq1UVQUFCaeqtWrRK1a9cWlpaWwsHBQQwfPlz/3JMnT8TQoUNFxYoVhZWVlfD09BS//PKL0ffL6N+QiEzPtm1pf2+6uMjynFi5UoihQ4Vo2VIIe3vD8zo7G9bt0EGIunWF6NFDiClThAgOFuLkSSFSXx6Sf7erVOn/bnd1NfzdnkynE+LhQyFOnxZi504hDhxIee75cyE8PISwsjJ+3o4dDc9jayuEg4MQPj5CdOkixKefCjF/vhAbNwpx5kzWrkHpxZlVGV1DU2MSYkROk5D0/lEBIdq3l3X278+4XvKxf3/KecuXT/t8dj19+lQ0adJEDBkyRNy7d0/cu3dP7Nu3TwAQdevWFX/88Ye4evWqePz4sQgLCxPLli0T586dE5cvXxaTJ08W1tbW4tatW/rzGUtCXFxcxIYNG8SVK1fEp59+KmxsbMTjx48zjS0hIUFMnTpVnDx5Uly/fl0EBweLEiVKiM2bN+vrfPfdd8La2loEBQWJ8PBwceLECf37a7Va8dprrwlPT0/xxx9/iGvXrolffvlF7N692+j7MQkhMh/bthm/sKtU8kidiERHC3HihBA//ijEpElCdOsmhJ+fYZ0WLdKez8lJiFathPjkEyG02tzFmjre9GLNDp1OiAcPhDh1SiYqixcL4e8vxOrVKXWePMn4utKxY86uQdnFJCSX8jMJ2bAhaz8AGzaknDcvkhAhhGjZsqUYNWqU/vH+/fsFALFjx45MX+vp6SkWLVqkf2wsCZk8ebL+8fPnzwUA8dtvv+Uo1uHDh4tu3brpHzs5OYlJkyYZrfv7778LCwsLER4enqVzMwkhMg/Z+at95EghHB3Trxcbm3Le5cuFmDxZiHXrjLdq5IaxVhtX19wlIFn1aqKyY4dMVL74Qoh+/WTiNW1azq5B2ZWdJIRjQvLQ8+fpP5fcd5mdfsNkN2/mOKQs8fHxMXj8/PlzTJ8+Hbt27cK9e/eQlJSEly9f4vbt2xmep27duvr7JUuWRKlSpfAgi6O1lixZgh9++AG3b9/Gy5cvkZCQAG9vbwDAgwcPcPfuXbRu3droa8PCwuDi4oLq1atn6b2IyDwcOpT1GSfPnsnxIgDg4ADUrAnUqJFy++r4kaFD8y/mrl2Bd99VZsVUlQqoUEEeDRoYr3PgQNbOldVrVW4xCclDWZks0ry5HFAUGYk0g4IA+UPk4iLrZee8uZF6lsu4ceOwd+9efP3116hatSo0Gg26d++OhISEDM9TvHhxg8cqlQo6nS7T99+0aRPGjRuHb775Bk2aNIGtrS2++uorHD9+HACg0WgyfH1mzxOReXj+XP6+S17UeN68rL3u3j1g3Djgk09kwmFnl38xZoVaLWe4mKKcXIPyk+LrhCxZsgRubm6wtraGr68vTpw4kW7dxMREzJgxAx4eHrC2tka9evWwZ88egzparRZTpkyBu7s7NBoNPDw8MHPmTAhj37YC1Go5BQpI+Y+WLPlxUFD+ZM2WlpbQZmEi+JEjRzBw4EB06dIFXl5ecHBwwM18bI45cuQImjZtimHDhqF+/fqoWrUqrl27pn/e1tYWbm5uCA0NNfr6unXr4s6dO7h8+XK+xUhEeev5c9laEBQkZ3fUrg2UKgVcv55Sx9U1a+dydAQ8PYHGjZVPQEydktcgYxRNQjZv3oyxY8di2rRpOH36NOrVqwc/P790m/AnT56M5cuXY9GiRbhw4QI+/vhjdOnSBWfOnNHXmTdvHpYuXYrFixfj4sWLmDdvHubPn49FixYV1MfKVNeucgqUs7NhuYtL/k6NcnNzw/Hjx3Hz5k08evQo3VaKatWqISQkBGFhYTh79iz69u2bpRaNnKpWrRr++ecf/P7777h8+TKmTJmCkydPGtSZPn06vvnmGyxcuBBXrlzB6dOn9f+mLVu2RIsWLdCtWzfs3bsXN27cwG+//ZYmQSUi5W3ZkpJwtGgBjBkDBAcDFy/Kv8z/97+UujNnygQjve2eVCqZqBTUX+2FhVLXIKNyPvQk9xo3bmwwzVKr1QonJycREBBgtL6jo6NYvHixQVnXrl1Fv3799I/feecdMXjw4AzrZCanA1OzKylJjkDesEHe5mZKVFaEh4eL1157TWg0GgGkTNF98uSJQb0bN26IVq1aCY1GI1xdXcXixYvTDGo1NjB1+/btBuexs7MTq18dup2OuLg4MXDgQGFnZydKly4tPvnkE+Hv7y/q1atnUG/ZsmWiRo0aonjx4sLR0VGMHDlS/9zjx4/FoEGDRLly5YS1tbWoU6eO+PXXX42+HwemEuWfmBghDh4UIjBQDoisWVOIP/5Ief6nnwynw3bqJMSXXwrx669C3LuX9nz5OeOkqMuva5BZzI6Jj48XarU6zYWrf//+olOnTkZfU7ZsWfH9998blPXr109UrlxZ/3j27NmicuXK+pkSYWFhomLFiiI4ODjdWOLi4kR0dLT+iIiIKJAkhJTBf0OiFHlxITp3Toi+fYWoUcP4dNrZs1PqPnwoxK5dQty/n/XzKznjhLLPLGbHPHr0CFqtFvb29gbl9vb2uHTpktHX+Pn5ITAwEC1atICHhwdCQ0MREhJiMM7B398fMTExqFmzJtRqNbRaLWbPno1+/fqlG0tAQAC+/PLLvPlgRERmIjtLd8fEAGfOAKdOyaNjR6B3b/lcUhKwYYPhORo2TDkaN055rnx5oH377MWp5IwTyl9mNTtmwYIFGDJkCGrWrAmVSgUPDw8MGjQIP/zwg77Oli1bsH79emzYsAGenp4ICwvD6NGj4eTkhAEDBhg974QJEzB27Fj945iYGLhmdUQUZerjjz9GcHCw0efee+89LFu2rIAjIqLMlu5euxZ4+DAl6bh82bCulVVKEuLpKcdvJCcdFSvmfbymPOOEck6xJKR8+fJQq9WIiooyKI+KioKDg4PR11SoUAE7duxAXFwcHj9+DCcnJ/j7+6NKlSr6OuPHj4e/vz96////Di8vL9y6dQsBAQHpJiFWVlawsrLKo09Gqc2YMQPjxo0z+lypUqUKOBoiymzbeZUKmDAh7Z4prq4piUarVinlxYsDkyfnb8xUOCmWhFhaWqJhw4YIDQ1F586dAQA6nQ6hoaEYMWJEhq+1traGs7MzEhMTsW3bNvTs2VP/3IsXL2BhYTjpR61W5+vsDspYxYoVUTE//jQiohz566/MFwGLjAQ6dZIbpPn4yMWv+N+Y8pqi3TFjx47FgAED4OPjg8aNGyMoKAixsbEYNGgQAKB///5wdnZGQEAAAOD48eOIjIyEt7c3IiMjMX36dOh0Onz++ef6c3bs2BGzZ89GpUqV4OnpiTNnziAwMBCDBw9W5DMSEZmSGTOAJUuyVrd3b6BPn/yNh4o2RZOQXr164eHDh5g6dSru378Pb29v7NmzRz9Y9fbt2watGnFxcZg8eTKuX78OGxsbtG/fHuvWrUPp0qX1dRYtWoQpU6Zg2LBhePDgAZycnPDRRx9h6tSpBf3xiIgUJQRw7hzwyo4KOHsWyOJuCgW2dDcVXSohTGQpURMSExMDOzs7REdHpxmzEBcXhxs3bsDd3R3W1tYKRUi5wX9DKsyEAE6eBH76SS48dfMmEB4OJG+tdOSI3G9l3Djg7t2Ml+6+cYMzUCj7MrqGpmZWs2OIiCgtIYATJ1ISj1u3Up4rUQI4fz4lCWnWTN5aWspZMCqVYSKixNLdVHQxCSEiMnO7dwMdOqQ8LllSPu7ZE3j7bZmIpJa8dLexdUKCggp46W4qspiEKEmrNavVd9zc3DB69GiMHj1a6VCIiiSdDjh2TLZ4uLnJBAIA3nxT/gpp2RLo0QNo1w7IyubSXASMlMYkRCnZWaqQiIosnQ44ejSlqyV57Y4aNYBPP5XdJxqNHOeRk+SBi4CRkpiEKCGzpQoLfBtDIjJF06YBq1YZLhpmayvX7+jRI2VhMYCtF2SeLDKvQpkSAoiNzdoREyP/fElvqUJAtpDExGR+rmxMbFqxYgWcnJzSLNr27rvvYvDgwbh27Rreffdd2Nvbw8bGBo0aNcK+ffty/JUEBgbCy8sLJUuWhKurK4YNG4bnz58b1Dly5AjeeOMNlChRAmXKlIGfnx+ePHkCQC5cN3/+fFStWhVWVlaoVKkSZs+eneN4iEydVgv8/bfhf+vbt2UCUqoU8N57wM6dcnptcLDsRrHgb3Ayc/wRzgsvXgA2Nlk77OzSroX8KiFkF42dXebnevEiyyH26NEDjx8/xv79+/Vl//33H/bs2YN+/frh+fPnaN++PUJDQ3HmzBm8/fbb6NixI27fvp2jr8TCwgILFy7Ev//+i7Vr1+LPP/80WFQuLCwMrVu3Ru3atXH06FEcPnwYHTt21G9GOGHCBMydOxdTpkzBhQsXsGHDhjSbHRKZMq0WOHAA2LhR3r6yz6ZBnYMHgREjZG9ss2Zyn5Zko0YBP/8sE49162QLCGeVU6GS73v6mqGMtiE2ug388+dp964uiOP582x9rnfffVcMHjxY/3j58uXCyclJaLVao/U9PT3FokWL9I8rV64svv3222y9Z7KffvpJlCtXTv+4T58+olmzZkbrxsTECCsrK7Fy5cocvVdmjP4bEuUhY1vPu7jI8qQkIfbvF2LYMCEcHAzrlC4txObNSkdPlDsZXUNT45iQvFCiBJCqqyFdf/2VtX2sd+8GWrTI/H2zoV+/fhgyZAi+++47WFlZYf369ejduzcsLCzw/PlzTJ8+Hbt27cK9e/eQlJSEly9f5rglZN++fQgICMClS5cQExODpKQkxMXF4cWLFyhRogTCwsLQo0cPo6+9ePEi4uPj0bp16xy9N5GSMhvyNW0aMH16Snnp0kDnznKMR5s2cv0OoqKCSUheUKnkxPysaNtWtrtGRma8VGHbtnk+0qxjx44QQmDXrl1o1KgRDh06hG+//RYAMG7cOOzduxdff/01qlatCo1Gg+7duyMhISHb73Pz5k106NABn3zyCWbPno2yZcvi8OHD+OCDD5CQkIASJUpAk8H8wYyeIzJlWdmd9vvv5cJhzZrJxKN1ayYeVHRxTEhBU6vlNFwgZVh7snxeqtDa2hpdu3bF+vXrsXHjRtSoUQMNGjQAIAeJDhw4EF26dIGXlxccHBxw8+bNHL3PqVOnoNPp8M033+C1115D9erVcffuXYM6devWRWhoqNHXV6tWDRqNJt3niUzVoUOZ70575w6wbBnwww9yPQ8mIFSUMQlRQvJShc7OhuUuLvk+Pbdfv37YtWsXfvjhB/Tr109fXq1aNYSEhCAsLAxnz55F375908ykyaqqVasiMTERixYtwvXr17Fu3TosW7bMoM6ECRNw8uRJDBs2DP/73/9w6dIlLF26FI8ePYK1tTW++OILfP755/jxxx9x7do1HDt2DKtWrcrVZyfKT3/+CYwfn7W69+/nbyxE5oJJiFK6dpU7S+3fD2zYIG9v3Mj39UHefPNNlC1bFuHh4ejbt6++PDAwEGXKlEHTpk3RsWNH+Pn56VtJsqtevXoIDAzEvHnzUKdOHaxfvx4BAQEGdapXr44//vgDZ8+eRePGjdGkSRPs3LkTxYrJHsIpU6bgs88+w9SpU1GrVi306tULD7K69SeRArZvB/75J2t1uTstkcRddI3gLrqFG/8NKTcSE4Fff5VjO/z95TLngNwkbsUKYMsWOaWWu9NSUcVddImI8ti1azLxWLMmpTulbNmUJKROHWDhQrkEOnenJcoadsdQtq1fvx42NjZGD09PT6XDI8ozWi2waZOcwVK1KjB3rkxAKlYEPv8cmDo17WsUHPJFZHbYEkLZ1qlTJ/j6+hp9rnjx4gUcDVH+UamASZOA69flfT8/YMgQoGNHIKMfde5OS5Q1TEIo22xtbWFra6t0GER56sULuVPtTz8B27YBVlZyb5YvvpDL+gweDFSunPXzcXdaoswxCckhjuc1X/y3o1edOQOsXAmsXy/3jQSAHTuAXr3k/aFDFQuNqNBjEpJNyd0NL1684MqeZip5FVg128aLrGfP5Mz4lSsNN4xzdwc+/DDzHROIKG8wCckmtVqN0qVL69esKFGiBFSpVz4lk6XT6fDw4UOUKFFCvyYJFT137gAffyzvW1oCXbrIsR6tWskuGCIqGPwtnAMODg4AwMWzzJSFhQUqVarE5LGI+O8/IDgYiIoCZs+WZbVqAQMHAl5eQP/+QPnyioZIVGRxsTIjsrrQilarRWJiYgFGRnnB0tISFvxz12xptZnPOhECOHhQdrds2wbEx8sWj8hIJhxE+Y2LlRUQtVrNcQVEBSgkRO5S++omcS4uck/Irl3lSqVr1shFxa5cSalTr57sbuECuUSmhUkIEZmFkBC5EmnqttvISFm+datcDv2LL2S5jQ3Qt69MPho2TLtpNREpj0kIEZk8rVa2gBjrPBZCJhijRwMnTsjptQMHyim2NjYFHCgRZQuTECIyeYcOGXbBpCYEEBEBXLok6xKReeDoPCIyeffu5W09IjINTEKIyOQ9epS1eo6O+RsHEeUtdscQkcmzt8/4eZVKzpJp3rxg4iGivMGWECIyOUIAt2+nPO7RQ85yUanSznJJfhwUxF1qicwNkxAiMim3bwMdOwINGqR0w6hUwIoVchqus7NhfRcXWd61a8HHSkS5w+4YIjIJWi2weDEwaRIQGytXOD18GOjcOaVO167Au+9mvmIqEZkHJiFEpLizZ+Xutf/8Ix+//rps+ahVK21dtRp4440CDY+I8gm7Y4hIMUIAEybIFU3/+QewswOWL5f7vhhLQIiocGFLCBEpRqUCnjyRXTE9esg9YDjNlqjoYBJCRAXq0SMgLk4OKAWAuXOBDh3kQURFC7tjiKhACAGsWwfUrAkMHpyyD0zp0kxAiIoqJiFElO+uXQPatgX69wcePwbu35e3RFS0MQkhonyTmAjMnw94eQH79gFWVsCcOcCpU0D58kpHR0RK45gQIsoXt27JNT3OnpWP33wTWLYMqFZN2biIyHSwJYSI8oWDAxAfD5QtC6xZI1tCmIAQ0avYEkJEeebPP+UKpsWLy66Xn34CKlaUBxFRamwJIaJcu38f6NULaN1abiSXrE4dJiBElD4mIUSUY0IA338vVzfdsgWwsACePVM6KiIyF4onIUuWLIGbmxusra3h6+uLEydOpFs3MTERM2bMgIeHB6ytrVGvXj3s2bMnTb3IyEi89957KFeuHDQaDby8vPBP8qYURJQnwsOBVq2AIUOAp0/lrrcnTwIzZigdGRGZC0WTkM2bN2Ps2LGYNm0aTp8+jXr16sHPzw8PHjwwWn/y5MlYvnw5Fi1ahAsXLuDjjz9Gly5dcObMGX2dJ0+eoFmzZihevDh+++03XLhwAd988w3KlClTUB+LqNALDgbq1pV7vJQoAXzzDXD8uExEiIiySiVE8rqFBc/X1xeNGjXC4sWLAQA6nQ6urq4YOXIk/P3909R3cnLCpEmTMHz4cH1Zt27doNFoEBwcDADw9/fHkSNHcOjQoRzHFRMTAzs7O0RHR6NUqVI5Pg9RYXXhAuDtLceALF0KuLkpHRERmYrsXEMVawlJSEjAqVOn0KZNm5RgLCzQpk0bHD161Ohr4uPjYW1tbVCm0Whw+PBh/eOff/4ZPj4+6NGjBypWrIj69etj5cqVGcYSHx+PmJgYg4OIUkRHA9u2pTyuXRs4cwbYvZsJCBHlnGJJyKNHj6DVamFvb29Qbm9vj/v37xt9jZ+fHwIDA3HlyhXodDrs3bsXISEhuHfvnr7O9evXsXTpUlSrVg2///47PvnkE3z66adYu3ZturEEBATAzs5Of7i6uubNhyQyE1otcOAAsHGjvNVqU57bsUMmHT17Aq8OrfL0lLvgEhHllOIDU7NjwYIFqFatGmrWrAlLS0uMGDECgwYNgoVFysfQ6XRo0KAB5syZg/r162Po0KEYMmQIli1blu55J0yYgOjoaP0RERFREB+HyCSEhMjWjFatgL595a2bG7ByJdC1K9ClC3D3LlClilyGnYgoryiWhJQvXx5qtRpRUVEG5VFRUXBwcDD6mgoVKmDHjh2IjY3FrVu3cOnSJdjY2KBKlSr6Oo6Ojqhdu7bB62rVqoXbt2+nG4uVlRVKlSplcBAVBSEhQPfuwJ07huV37gBDhwLbtwPFigGTJgH/+x/QpIkycRJR4aRYEmJpaYmGDRsiNDRUX6bT6RAaGoommfyms7a2hrOzM5KSkrBt2za8++67+ueaNWuG8PBwg/qXL19G5cqV8/YDEJk5rRYYNUqu9ZEeS0s57XbWLECjKbjYiKhoUHTZ9rFjx2LAgAHw8fFB48aNERQUhNjYWAwaNAgA0L9/fzg7OyMgIAAAcPz4cURGRsLb2xuRkZGYPn06dDodPv/8c/05x4wZg6ZNm2LOnDno2bMnTpw4gRUrVmDFihWKfEYiU3XoUNoWkNQSEuQaIERE+UHRJKRXr154+PAhpk6divv378Pb2xt79uzRD1a9ffu2wXiPuLg4TJ48GdevX4eNjQ3at2+PdevWoXTp0vo6jRo1wvbt2zFhwgTMmDED7u7uCAoKQr9+/Qr64xGZtFfGc+dJPSKi7FJ0nRBTxXVCqChYtgz45JPM6+3fD7zxRr6HQ0SFhFmsE0JEyrh/Hxg8OPMERKUCXF3lrrhERPmBSQhREZGQAHz9NVC9OrB6tSxr2VImG6nX+0h+HBQEqNUFGiYRFSFMQoiKiCNHgPHj5S63Pj7A33/Lhcm2bgWcnQ3rurjI8q5dFQmViIoIjgkxgmNCqLCIjQVKlkx5PGSIXOtj4EDglTHf0GrlbJl79wBHR9kFwxYQIsqJ7FxDFZ0dQ0T549kzubbHDz8A584Byev/pbeNklrNwadEVPDYHUNUiOh0wNq1ctzH/PnAo0dyPxgiIlPElhCiQuLECWDkSHkLANWqAd9+C7zzjrJxERGlh0kIkZkTQu7z8v338rGNDTB1qlyS3dJS2diIiDLCJITIzKlUMvEAgAEDgIAAObiUiMjUMQkhMkO7dgGVKgFeXvLxtGlA796Ar6+ycRERZQcHphKZkfBwoH17oEMHYMSIlB1wS5dmAkJE5odJCJEZiImRC415eQG//QYULw689hqQmKh0ZEREOcfuGCITljzldsIEICpKlnXoAAQGytkvRETmjEkIkQnbtEluNgfItT+CgoB27RQNiYgoz7A7hsjE6HQp93v2BJo2lRvPnTvHBISIChe2hBCZiPh4ubjY5s3AsWOAlRVQrBhw+HDaXW6JiAoDtoQQKUwI4JdfAE9POfYjLAxYvz7leSYgRFRYMQkhUtClS7KLpVMn4No1udHc2rVyl1siosKO3TFE+UirBQ4dAu7dk6uYNm8ud6xNSgI+/xxYtEjet7QExowBJk0CbG2VjpqIqGAwCSHKJyEhcv+WO3dSylxcgAULgK5d5cJjSUlAx45yym3VqsrFSkSkBJUQyWsuUrKYmBjY2dkhOjoapUqVUjocMkMhIUD37ikrmiZLHt+xdatceOzaNeDttws+PiKi/JKdayhbQojymFYrW0CMpfdCyERk9Gjgxg0uOEZERRsHphLlsUOHDLtgUhMCiIiQ9YiIijImIUR57N69vK1HRFRYMQkhymMVK2atnqNj/sZBRGTqmIQQ5bEaNQCLDP5nqVSAq6ucrktEVJQxCSHKYy4uwOzZ8n7q1U6THwcFyfVCiIiKMiYhRHlg/35g586Ux/7+wLZtgLOzYT0XFzk9t2vXgo2PiMgUcYouUS4tWwaMHClXPT1xQu4BA8hE4913ja+YSkRETEKIciwpSS61vnixfNyrF+DhYVhHrQbeeKPAQyMiMgtMQohy4MkToGdPYN8++XjOHNkFwx1viYiyjkkIUTZdviz3e7l8GShZEggOBjp3VjoqIiLzwySEKJvWrJEJiKsr8MsvQL16SkdERGSemIQQZdOMGXJ/mLFjAXt7paMhIjJfnKJLlInERODbb4GEBPm4WDFg3jwmIEREucWWEKIM/Pcf0KMH8OefwKVLwPLlSkdERFR4MAkhSsfFi0CnTsDVq4CNDfDOO0pHRERUuDAJITJizx657kdMDODmBvz8M+DlpXRURESFC8eEEL1CCGDBAtnqERMDvP66XAWVCQgRUd5jEkL0ivv3galTAZ0OGDRILkZWoYLSURERFU7sjiF6haMjsHkzcP488NlnXAGViCg/MQmhIu/CBeDxY7m5HAC8/bY8iIgof7E7hoq03buB116Tu91evap0NERERQuTECqShAACA+UeMM+eyYGnpUsrHRURUdHCJISKnPh44MMP5ZgPnU7e37sXKF9e6ciIiIoWk0hClixZAjc3N1hbW8PX1xcnTpxIt25iYiJmzJgBDw8PWFtbo169etizZ0+69efOnQuVSoXRo0fnQ+Rkbh4+BNq0AX74AbCwAIKCgBUrAEtLpSMjIip6FE9CNm/ejLFjx2LatGk4ffo06tWrBz8/Pzx48MBo/cmTJ2P58uVYtGgRLly4gI8//hhdunTBmTNn0tQ9efIkli9fjrp16+b3xyAzMW8ecPgwUKoUsGsXMGoUZ8AQESlFJYQQSgbg6+uLRo0aYfHixQAAnU4HV1dXjBw5Ev7+/mnqOzk5YdKkSRg+fLi+rFu3btBoNAgODtaXPX/+HA0aNMB3332HWbNmwdvbG0FBQUZjiI+PR3x8vP5xTEwMXF1dER0djVKlSuXRJyVTEBcHfPABMHkyUKuW0tEQERU+MTExsLOzy9I1VNGWkISEBJw6dQpt2rTRl1lYWKBNmzY4evSo0dfEx8fD2traoEyj0eDw4cMGZcOHD8c777xjcO70BAQEwM7OTn+4urrm4NOQKRICCAmRYz8AwNoaWL+eCQgRkSlQNAl59OgRtFot7FPtiW5vb4/79+8bfY2fnx8CAwNx5coV6HQ67N27FyEhIbh3756+zqZNm3D69GkEBARkKY4JEyYgOjpaf0REROT8Q5HJiI+Xq5526yZbPoiIyLSY3WJlCxYswJAhQ1CzZk2oVCp4eHhg0KBB+OGHHwAAERERGDVqFPbu3ZumxSQ9VlZWsLKyys+wqYA9eAB06QL8/TegVgNOTkpHREREqSnaElK+fHmo1WpERUUZlEdFRcHBwcHoaypUqIAdO3YgNjYWt27dwqVLl2BjY4MqVaoAAE6dOoUHDx6gQYMGKFasGIoVK4aDBw9i4cKFKFasGLRabb5/LlLW//4HNGokExA7O+C334ARI5SOioiIUlM0CbG0tETDhg0RGhqqL9PpdAgNDUWTJk0yfK21tTWcnZ2RlJSEbdu24d133wUAtG7dGufOnUNYWJj+8PHxQb9+/RAWFga1Wp2vn4mUtXMn0LQpcPs2UK0acPw48NZbSkdFRETGKN4dM3bsWAwYMAA+Pj5o3LgxgoKCEBsbi0GDBgEA+vfvD2dnZ/34juPHjyMyMhLe3t6IjIzE9OnTodPp8PnnnwMAbG1tUadOHYP3KFmyJMqVK5emnMyXVgscOgTcuyc3nWveXO7/0rcv8OKFXAtkyxagTBmlIyUiovQonoT06tULDx8+xNSpU3H//n14e3tjz549+sGqt2/fhoVFSoNNXFwcJk+ejOvXr8PGxgbt27fHunXrUJprbhcZISFyfY87d1LKXFyABQuA778HjhwBvv0WKF5cuRiJiChziq8TYoqyM8eZClZICNC9u5x6+6rkBce2bgW6di34uIiISDKbdUKIskOrlS0gxtLm5LLRo2U9IiIyfTlKQrp164Z58+alKZ8/fz569OiR66CIjDl0yLALJjUhgIgIWY+IiExfjpKQv/76C+3bt09T3q5dO/z111+5DorImFfWo8uTekREpKwcJSHPnz+HpZFtR4sXL46YmJhcB0VkjKNj3tYjIiJl5SgJ8fLywubNm9OUb9q0CbVr1851UETGNG8OVKiQ/vMqFeDqKusREZHpy9EU3SlTpqBr1664du0a3nzzTQBAaGgoNm7ciJ9++ilPAyRKplbLJOPhw7TPJc+OCQqS9YiIyPTlqCWkY8eO2LFjB65evYphw4bhs88+w507d7Bv3z507tw5j0MkSnHokJyim3ovGBcXTs8lIjI3XCfECK4TYvqMrZjKFhAiIuVl5xqao+6YkydPQqfTwdfX16D8+PHjUKvV8PHxyclpiYx68ADYsAEYOTIl0VCrgTfeUDQsIiLKpRx1xwwfPhwRERFpyiMjIzF8+PBcB0WUTAhgyBBgzBiAP1pERIVLjpKQCxcuoEGDBmnK69evjwsXLuQ6KKJkP/wA/PwzYGkJDBumdDRERJSXcpSEWFlZISoqKk35vXv3UKyY4nviUSFx/bpchh0AZs0C6tZVNBwiIspjOUpC2rZtiwkTJiA6Olpf9vTpU0ycOBFvvfVWngVHRZdWC/TvDzx/DrRoAYwdq3RERESU13LUbPH111+jRYsWqFy5MurXrw8ACAsLg729PdatW5enAVLR9NVXwJEjgK0tsHYtZ74QERVGOZ6iGxsbi/Xr1+Ps2bPQaDSoW7cu+vTpg+LFi+d1jAWOU3SV9d9/QOXKshVk9Wpg4EClIyIioqzK9ym6AFCyZEm8/vrrqFSpEhISEgAAv/32GwCgU6dOOT0tEcqWla0g69cDAwYoHQ0REeWXHCUh169fR5cuXXDu3DmoVCoIIaBKXjcbgFarzbMAqWiqW5cDUYmICrscDUwdNWoU3N3d8eDBA5QoUQLnz5/HwYMH4ePjgwMHDuRxiFRUHDkC/POP0lEQEVFByVEScvToUcyYMQPly5eHhYUF1Go1Xn/9dQQEBODTTz/N6xipCHj6FOjTB2jSBNi9W+loiIioIOQoCdFqtbC1tQUAlC9fHnfv3gUAVK5cGeHh4XkXHRUZI0cCERGAm5uckktERIVfjsaE1KlTB2fPnoW7uzt8fX0xf/58WFpaYsWKFahSpUpex0iF3JYtQHAwYGEBrFsH2NgoHRERERWEHCUhkydPRmxsLABgxowZ6NChA5o3b45y5cph8+bNeRogFW6RkcDHH8v7EycCr72mbDxERFRwcrxOSGr//fcfypQpYzBLxlxxnZCCIQTw9tvAH38ADRsCR48ChWCZGSKiIq1A1glJrWzZsnl1Kioitm+XCYi1teyGYQJCRFS0cLc5UkznzsDChTL5qFVL6WiIiKigMQkhxVhYyFkxRERUNOVoii5Rbvzyi9wXhoiIijYmIVSgjh8HunQBvL2Bx4+VjoaIiJTEJIQKTGws8P77gFYL+PoC5copHRERESmJSQgVmPHjgStXABcXYPFipaMhIiKlMQmhAvHbb8DSpfL+mjVAmTKKhkNERCaASQjlu0ePgMGD5f1Ro4DWrZWNh4iITAOTEMp3/v7A/ftyLZCAAKWjISIiU8F1QijfzZolZ8JMmQJoNEpHQ0REpoJJCOU7Bwe5RDsREdGr2B1TELRa4MABYONGeavVKh1RvtPpgD//VDoKIiIyZUxC8ltICODmBrRqBfTtK2/d3GR5Ifbtt3IA6vDhSkdCRESmiklIfgoJAbp3B+7cMSyPjJTlhTQROX8emDhR3vf2VjQUIiIyYUxC8otWK+ejCpH2ueSy0aMLXddMfDzw3ntAQgLQoQPw4YdKR0RERKaKSUh+OXQobQvIq4QAIiJkvUJk2jTg7FmgfHng++8BlUrpiIiIyFQxCckv9+7lbT0zcOgQMH++vL9iBWBvr2w8RERk2piE5BdHx7ytZ+Li44EBA2QDz6BBcqdcIiKijDAJyS/Nm8ud2jLrj1i4UC4nauasrOSmdK+9BgQFKR0NERGZAyYh+UWtBhYskPdTJyLJjy0s5CpetWvLXd2MDWI1I+3bA3//DZQqpXQkRERkDkwiCVmyZAnc3NxgbW0NX19fnDhxIt26iYmJmDFjBjw8PGBtbY169ephz549BnUCAgLQqFEj2NraomLFiujcuTPCw8Pz+2Ok1bUrsHUr4OxsWO7iAmzbBpw+DTRoADx5Ivsw2rUDbt0q+Dhz4cEDOb42GQeiEhFRVimehGzevBljx47FtGnTcPr0adSrVw9+fn548OCB0fqTJ0/G8uXLsWjRIly4cAEff/wxunTpgjNnzujrHDx4EMOHD8exY8ewd+9eJCYmom3btoiNjS2oj5Wia1fg5k1g/35gwwZ5e+OGLK9XDzh+HJg7V/Zn/P474Okp+zV0uoKPNZuEAD74APDyAnbtUjoaIiIyNyohlO0D8PX1RaNGjbB48WIAgE6ng6urK0aOHAl/f/809Z2cnDBp0iQMf2Upzm7dukGj0SA4ONjoezx8+BAVK1bEwYMH0aJFi0xjiomJgZ2dHaKjo1GqoPoWwsPlohqHD8vHr78u57jWqFEw758DK1cCQ4cClpbAqVNAnTpKR0RERErLzjVU0ZaQhIQEnDp1Cm3atNGXWVhYoE2bNjh69KjR18THx8Pa2tqgTKPR4HDyxduI6OhoAEDZsmXTPWdMTIzBUeBq1AAOHpStICVLymSkXj1g3jwgKang48nEtWvAmDHy/pw5TECIiCj7FE1CHj16BK1WC/tUC0rY29vjfjozRvz8/BAYGIgrV65Ap9Nh7969CAkJwb101tvQ6XQYPXo0mjVrhjrpXCkDAgJgZ2enP1xdXXP3wXLKwkJutvLvv0DbtnLeq78/4OsrVwAzEUlJwPvvA7GxwBtvpCQjRERE2aH4mJDsWrBgAapVq4aaNWvC0tISI0aMwKBBg2BhYfyjDB8+HOfPn8emTZvSPeeECRMQHR2tPyJeHWmphMqVgT175IyZMmXkAFYfH2DKFJmYKGz+fODoUTkLZs0amTsRERFll6KXj/Lly0OtViMqKsqgPCoqCg4ODkZfU6FCBezYsQOxsbG4desWLl26BBsbG1SpUiVN3REjRuDXX3/F/v374eLikm4cVlZWKFWqlMGhOJVKrv514YIcxJqUBMyaBdSvDxw7plhYZ8/KpdkB2XNUubJioRARkZlTNAmxtLREw4YNERoaqi/T6XQIDQ1FkyZNMnyttbU1nJ2dkZSUhG3btuHdd9/VPyeEwIgRI7B9+3b8+eefcHd3z7fPkO8cHOR03q1b5TroFy8CTZvKPhAFZvtUry57jHr0kBvVERER5ZTis2M2b96MAQMGYPny5WjcuDGCgoKwZcsWXLp0Cfb29ujfvz+cnZ0REBAAADh+/DgiIyPh7e2NyMhITJ8+HTdu3MDp06dRunRpAMCwYcOwYcMG7Ny5EzVemV1iZ2cHjUaTaUyKzI7Jiv/+k8nHjz/Kx+7ucopK69YFHopWK9djIyIiepXZzI4BgF69euHrr7/G1KlT4e3tjbCwMOzZs0c/WPX27dsGg07j4uIwefJk1K5dG126dIGzszMOHz6sT0AAYOnSpYiOjsYbb7wBR0dH/bF58+aC/nh5q2xZYO1aYPduwNVVrjfSpg0wZAjw9Gm+vvWNGzLxSMYEhIiIckvxlhBTZLItIa969kzOnPnuO/nYyQlYuhTo1CnP3+rJE6BuXcDNDdiypdDsuUdERPnArFpCKIdsbYElS+TaItWqAXfvAu++C/TpAzx8mKdvNWIEcOcOEBXFfWGIiCjvMAkxdy1ayCkrn38u58pu2iQ3xNu4MU82xNu0Sa42r1YD69bJddSIiIjyApOQwkCjkSurHj8uN3J59Ajo21d2zURG5vi0kZHAJ5/I+5MnyzXTiIiI8gqTkMLExwf45x/gyy+B4sWBX3+VrSIrV2a5VUSrBQ4cANavl707T5/K006alK+RZ19yoBs3yttXR80SEZFZYBJS2FhaAlOnAmfOAI0bAzExcpe51q3lhi8ZCAmRg09btZJrgJw6Jcv795c5jcl4NdC+feWtm5ssJyIis8EkpLDy9AT+/hsIDJTdNfv3y66awECjrQYhIUD37nIAamqjRpnQ9T29QCMjZbnJBEpERJnhFF0jzGKKbnZcuybXEtm/Xz729QVWrZKJCmRO4uZmPAEB5AryLi5yrRBF1wcxm0CJiIouTtElQx4eQGgosGKFnGN7/Ljcg2bmTCAhAYcOpX9dB+RwkogI4NChggsZgNwvJyJC7pWzdatskjHJQImIKCeKKR0AFRCVSraGtGsnp7z8+qscO7J1K+J7rgLgAwCwgBbNcQiOuId7cMQhNIcOslXhlYVrc+/lS9mFcudOyu2r9yMjgfv3AZ0u++fO00CJiCi/MAkpalxcgJ9/lrNKPv0U+N//0Pa8L+ZiHM7AG1/hc7gipbUhAi4YhQXYjq5ZWylVCCA6OuPk4s4duQ9OVhQrJleDdXaWg24PHsz8NbduyThUqqy9BxERKYJjQowodGNC0vPggezi2LQJACD+/3i1j04HeSH/uNxWLL3XGerHD4wnFa/ef/Eia+9fooRMipydDW9fvV+xolyEDUgZExIZmfmU4/r1gYkTgS5dOD6EiKgAZecayiTEiCKThCTbvh26bt1hIYx3fQgAwkINCxWyvh5H2bLGE4xXEw07u+y3ViTPjgEME5Hk83TsKMe/xMbKxzVryj12+vY1sXnGRESFE5OQXCpqSUjSvgMo9larrFW2sAAcHDJOMJyd5bTg/BISknaQqqsrEBQEdO0KPH4MLFwoj+Tdhd3c5NL2gwYB1tb5F5u50mrlgN579+QOhc2bswWJiHKESUguFbUk5ODHG9Fyed/MKy5aBHz8sRynobSsXDRjYuTOwoGBsusJkHU/+wz46CPAxqbg4zZFxpI6FxdgwQKZ1BERZQOTkFwqSkmIEMDgKgew+mYWWkL27wfeeCPfY8pzL17IdVG++kpO4QVkd9GoUcDIkUCZMsrGp6Tk7q3UvwaSu7e2bmUiQkTZwiQkl4pSEgIAd25pYVPHDXaxkVAZ+3EoLIuAJSQAwcFAQABw9aoss7UFhg0DxowB7O2Vja+gcfE3IsoHXKyMssWlshql1y6Q82BSDxRNfhwUZP4XIktLYPBg4NIlOUXZywt49kzuQOzmJqcsJ7eUFAV//cXF34hIUUxCirDkCSQAZJP71q1yUOmrXFwKX5O8Wg307g2EhQE7d8qN/uLi5JgXDw/ggw+AK1eUjjJ/PHgAbNgADByY9X9TLv5GRPmE3TFGFIXuGCGAhg3lOmCLFgHu7v//RFGcJSEE8OefwOzZKfvrWFgAPXvKtUa8vJSNLzfi4+VGhn/8Afz+u9xdObv27ZO7MBMRZQHHhORSUUhCdu0COnQASpaUC4yWK6d0RCbi6FGZjOzalVLWqRMwaZJsMTF1Qsjupj/+kMeBA2kXj/P2Btq2lYnF4MHA3bsZL/7WsqUcS+Pikp+RE1EhwSQklwp7EiIE0KSJ3Mdu/Hhg/nylIzJBYWHAnDmyKyr5v0ibNrJl5I03TGtJ+MeP5QJtyYlH6nEt9vYy6WjbFnjrLcMBuBkt/iYEYGUlW1PKlgW+/16uQEtElAEmIblU2JOQffvktcjaWk58cHBQOiITFh4OzJ0rWwKSkmRZkyayZaR9e2WSkcREubNwctJx8qRhAmFlJbvR/Pxk4uHllXGcGS3+5uUlV5v95x9Z/tFHct2VEiXy5aMRkfljEpJLhT0JadlSToz49FO5HhVlwc2bcp2RVatkywAguzUmTpQDPPNz3IwQwLVrckzHH3/IcSvPnhnWqVMnpbWjefPsJwkZjQVKSJA7Ls+fL2OpWVPuN1SvXt58PiIqVJiE5FJhTkL++ksmIZaW8rrGbv5sundPtgQsXZoyvahGDbk/Tb9+afenyelA36dP5WDZ5NaOGzcMny9fXjZnJXexpJ7VlB9CQ4H335efxdJSJiWffmpaXVNEpDgmIblUmJOQPn3kH7EffQQsW6Z0NGbs8WM5rWjhQuDJE1lWubLcn2bwYNnXlZ3l0JOSZLdKctJx/LjhZoHFiwPNmsmkw89PtsJYKDDD/tEjOYX555/l43btgNWri95Cb0SULiYhuVSYk5C4OHnNePvtV6blUs49eyZbRb75JmV/GgcHmSysW5fxcugNGqRMnQ0NBaKjDevWqJEyrqNlS9PZ60YImcGOHSt/oCpWBNaulT9URFTkMQnJpcKchFA+eflSjheZPz9rq64WK5Yy0DVZmTJyBk5yF0vlyvkTa145f142rZ0/Lx+PGSOXxLeyUjYuIlIUk5BcKoxJyH//AXZ2hX/dMcUlJACTJ8tBrJmxsACaNk0ZUOrjY37/QC9fAl98IbumADlYdeNGoFYtZeMiIsVw7xhKY+hQoHZtOTCV8pGlJVC/ftbqrlghB61OmQL4+ppfAgIAGo0cF/PLL3Kw7NmzcineFSsyXgCNiAhMQoqEf/8Ftm0DLl/myqgFwtExa/U8PPI3joLUoQPwv//JbqSXL+XI5+7dZRMcEVE6mIQUAXPmyNtu3QBPT2VjKRKaN5ezYNKbuqpSycXAmjcv2Ljym6MjsGcP8PXXcjZPSAhQt65cOp6IyAgmIYXclStySi4gF/mkAqBWp6wClzoRSX4cFGSe3S+ZsbAAPvtM7sFTrRoQGQm8+ab84UtMVDo6IjIxTEIKublzAZ1OtpZndagC5YGuXeU03NSLiLm4yPLU64QUNg0bAqdPyzVFhJDNcc2bA9evKx0ZEZkQzo4xorDMjrl5U/4xmpQktxrx9VU6oiIopyumFiY//SRHRj99CtjaAt99B7z3ntJREVE+4ewYAgBs3iwTkLfeYgKiGLVa7rrbp4+8LWoJCAD06CFnzbz+ulzc7f33ZRISE6N0ZESkMCYhhdjnn8uFOOfOVToSKvIqVZIb782YIROx9evl0vPHjikdGREpiElIIaZSyTGBDRooHQkR5CqxU6bIxWrc3OSmfK+/DsyebbhPDpExWq2cabVxo7zlz0yhwCSkEHryJGVPNSKT07QpEBYG9O4tLySTJwOtW2dtuXsqmkJCZOLaqhXQt6+8dXOT5WTWmIQUQvPmyW1HVq5UOhKidNjZARs2yI3vbGyAgwflku+8qFBqISFy4btXd6MG5PTv7t35M2PmmIQUMo8fA0uWyPF/WV24k0gRKhXQvz9w5ozcN+fJE7mi3tChQGys0tHlHrsPck+rBUaNMr4FQHLZ6NH8bs0Yk5BCZuFC4PlzOebvnXeUjoYoC6pWBY4cAfz9ZWKycqVMSsLClI4s59h9kDcOHUrbAvIqIWQ33m+/ca8iM8V1Qoww13VCoqNlN0x0tFwPq1s3pSMiyqbQUNk6cveu3Axw7lz5l7CFGf29lNx9kPpXa/JquUVhsbqcevZMbnZ1/ry83bdP3s+KEiXk4oBOTvIwdt/RUW66mN+K+PpA2bmGMgkxwlyTkDlz5OrYtWsD586Z1+9tIr1Hj4APPwR27pSP/fyANWsABwf52JR/wWu1ssUjvb/eVSq5au6NG6YTsxJevgQuXUpJOJKPW7fy/73Lls04UXFyAuzt5WyunAgJkYnzqz8DLi5yK4ciknwyCcklc0xCnj+Xv/seP5ZLMPTtq3RERLkgBLB8OTBmDBAXB1SsCKxeLe+byi/4uDg5juW//1Ju//5bjgzPzJYtQJcuOb/QmYvERODqVcNE4/x5WabTGX+NoyNQp448ateWf1k9fGi8uyU5qTt/HnjwQA5WvXtXHsbux8VlLW4LC5mIZJSoODvLhObV/aHYCgaASUiumWMSsmcP0LGjTEQuXiz8v9uoiPj3X7na7Llz6dfJzS94nU72X/73n2EykfrWWNnLlzn/XEDOL3R5LS9alnQ6uU9E6mQjPBxISDD+mjJlAC+vlISjTh25zXfZsob1ki/sgOHFPbv/7kLIrQMyS1Tu38/6QFdLy5R/Kycn+Yv4+XPjdYtQK5jZJSFLlizBV199hfv376NevXpYtGgRGjdubLRuYmIiAgICsHbtWkRGRqJGjRqYN28e3n777RyfMzVzTEIA+TsgMhJo1kzpSIjyUFwcMH48sHhx+nVUKnkB3bVLJhVZSSKePJEXpdz8CrSwAEqXlhfOMmVk2cmTWXtdei0Bqb16ocsoWbGxyX782e06EEJeqF9NNP79Vx4vXhh/j5Il0yYaderI7rWsJlfG4nR1lbtR53XLglYrW1WSk5P0kpVHj3J2/v375RYOhZhZJSGbN29G//79sWzZMvj6+iIoKAg//fQTwsPDUbFixTT1v/jiCwQHB2PlypWoWbMmfv/9d4wdOxZ///036v//NrHZPWdq5pqEEBVaBw7IGSb5pWTJlETi1VtjZa/eliplOPgqeUxIZGTG3QdXr8pE6NWLm7GLXXYudLa2mScqjo4yqQEy7zr44Qf5WVKP23j61Pj7W1kBtWqlJBnJR6VKeTNAzdTGAsXHy1iS/71+/hlYty7z15UvD7RrB7RoIT9D9er529KlALNKQnx9fdGoUSMs/v+/cnQ6HVxdXTFy5Ej4+/unqe/k5IRJkyZh+PDh+rJu3bpBo9EgODg4R+dMzZySkPh4+fvM01PpSIjy0caNWRvoVKqUvEBllDykLitTJuXCnBfyqvsAkP/B79/PvAvh2bOsx1e+vExKLl/O+hiJV6nV8sKZOtnw8Cja/cA5TZQrVpTJSHJSUreu2XfXZOcaquhPTEJCAk6dOoUJEyboyywsLNCmTRscPXrU6Gvi4+NhbW1tUKbRaHD48OFcnTM+Pl7/OMaMdvf88Ue5ttOwYXKRMqJCKasr7+3cqXxTd9euMtEw1s2R3e4DKys5775y5YzrPXtm2KKSXrKSkCBbV7LawuLgADRqZJhs1Kgh4yJDzZvLf+OMWsGcnOQ6OEeOyFad48dl18+2bfIAZCLdrFlKYuLjU6i/b0WTkEePHkGr1cLe3t6g3N7eHpcuXTL6Gj8/PwQGBqJFixbw8PBAaGgoQkJCoP3/gUQ5OWdAQAC+/PLLPPhEBSsxEQgIkPerVVM2FqJ8lZVf8C4usp4p6NoVePfdgus+sLWVyUGNGunXESKlC2jDhqzN4gkMlAODKXNqtRxL0727/Hk01gq2cKHsimnXTj6Oj5djiA4dkhs7HjkCxMTIxdd++03WsbYGfH3lz0/z5kCTJvLfu5Awu5UkFixYgGrVqqFmzZqwtLTEiBEjMGjQIFjkos9xwoQJiI6O1h8RZrKR1saNcqB1hQqyNYSo0Er+BQ+k7T9PfhwUZFrN2Gq1bJXp00feKh2bSgWUKyeb+1MN5E8X937InuRWMGdnw3IXF+PdcFZWcifpCRNk0vHkCXD6tPxZ7tZN/nKPi5N7K82aJdfMKVNGtk599hmwY0fOB8iaCEVbQsqXLw+1Wo2oqCiD8qioKDgkL0yUSoUKFbBjxw7ExcXh8ePHcHJygr+/P6pUqZLjc1pZWcHKzJq7tFq5OBkgfxZLlFA2HqJ8l5fdHEWdubUsmZPctIKp1UD9+vJI3jPn8mXZSnLokDxu3gT++UcegYHydbVrG44rcXXNWqymMNhXKKxx48ZixIgR+sdarVY4OzuLgICALL0+ISFBeHh4iAkTJuTZOaOjowUAER0dncVPUfA2bRICEKJsWSFiYpSOhqgAJSUJsX+/EBs2yNukJKUjMk/btgmhUslDXu7kkVy2bZvSEZIxt28LsX69EB99JETt2ob/dsmHm5sQ778vxMqVQly6JIROl/Y827YJ4eJi+DoXlzz5d8/ONVTxJGTTpk3CyspKrFmzRly4cEEMHTpUlC5dWty/f18IIcT7778v/P399fWPHTsmtm3bJq5duyb++usv8eabbwp3d3fx5MmTLJ8zM6aehGi1QtSpI39mZsxQOhoiMlvGLkSurkxAzMnDh0Js3y7E2LFC+PgIoVanTUoqVhSiWzchgoKEOH1aiJ9+Spt85mECmp1rqOLzqXr16oWHDx9i6tSpuH//Pry9vbFnzx79wNLbt28bjPeIi4vD5MmTcf36ddjY2KB9+/ZYt24dSpcuneVzmrsbN4CoKDmIeuRIpaMhIrNV0ANoKe+VLw907iwPQM6UOno0pfvm2LG0M3BSD5xNJoR8bvRo+XNRAD8Hiq8TYorMYZ2Q2Fjg7FmgaVOlIyEiIpMVHy/HjyTPwDl4MP3VbV+Vi5Vds3MNNbvZMSSVLMkEhIiIMmFlJdcd8fcHdu8GVqzI2uvu3cvfuP4fkxAzIgQQGpr1LSeIiIgMpJ4+nJ4Cmp7NJMSM/Pkn0KYN8NprWd/kkYiISC95enZ6+9WoVHKKbwFNz2YSYkZmzZK3vr4cN0ZERDlgYgv/MQkxE4cPy/2RihcHPv9c6WiIiMhsZXdl13yk+BRdyprkVpCBA7O+GB4REZFRJjI9m0mIGTh5Evj9d/mz4e+vdDRERFQoJO9vpCB2x5iB2bPlbb9+wP9vkUNERGT2mISYuJcv5R5TKhUwcaLS0RAREeUddseYOI0GOHECCAsDatRQOhoiIqK8w5YQM6BSyZ2diYiIChMmISbsl1+AmBiloyAiIsofTEJM1LVrclNENzfg4UOloyEiIsp7TEJMVECA3CPmtdeAChWUjoaIiCjvMQkxQbdvA2vXyvuTJysbCxERUX5hEmKC5s0DkpKAN98EmjZVOhoiIqL8wSTExNy9C6xaJe+zFYSIiAozJiEm5uuvgfh4oFkzxVfTJSIiyldMQkzMs2eAhQUwZUraXZaJiIgKEyYhJmblSuDqVaBtW6UjISIiyl9ctt0EubsrHQEREVH+Y0uIifj5Z+DyZaWjICIiKjhMQkxATAwwYABQqxZw9KjS0RARERUMJiEmYMkS4OlToHp1oHFjpaMhIiIqGExCFBYbCwQGyvsTJwJqtbLxEBERFRQmIQpbvhx49AioUgXo00fpaIiIiAoOkxAFxcUBX30l70+YABTjXCUiIipCmIQoaNUq4P59wNUV6N9f6WiIiIgKFpMQBalUQJkywBdfAJaWSkdDRERUsFRCCKF0EKYmJiYGdnZ2iI6ORqlSpfL1vZ49kwmIlVW+vg0REVGByM41lKMQFGZrq3QEREREymB3jAL++APYvRtgGxQRERVlTEIKmFYLjBoFvPOO3KyOiIioqGISUsBCQoBLl4DSpYHevZWOhoiISDkcE1IAtFrg0CEgMhKYOlWWjRoF5POYVyIiIpPGJCSfhYTIhOPOnZQylQpwd1cuJiIiIlPAJCQfhYQA3bunHYAqBDBokJwZ07WrMrEREREpjWNC8knyANSMZsCMHi3rERERFUVMQvLJoUOGXTCpCQFERMh6RERERRGTkHxy717e1iMiIipsmITkE0fHvK1HRERU2DAJySfNmwMuLnImjDEqldw9t3nzgo2LiIjIVDAJySdqNbBggbyfOhFJfhwUJOsREREVRUxC8lHXrsDWrYCzs2G5i4ss5/RcIiIqyhRPQpYsWQI3NzdYW1vD19cXJ06cyLB+UFAQatSoAY1GA1dXV4wZMwZxcXH657VaLaZMmQJ3d3doNBp4eHhg5syZEArtFte1K3DzJrB/P7Bhg7y9cYMJCBERkaKLlW3evBljx47FsmXL4Ovri6CgIPj5+SE8PBwVK1ZMU3/Dhg3w9/fHDz/8gKZNm+Ly5csYOHAgVCoVAgMDAQDz5s3D0qVLsXbtWnh6euKff/7BoEGDYGdnh08//bSgPyIA2eXyxhuKvDUREZHJUgmlmggA+Pr6olGjRli8eDEAQKfTwdXVFSNHjoS/v3+a+iNGjMDFixcRGhqqL/vss89w/PhxHD58GADQoUMH2NvbY9WqVfo63bp1g0ajQXBwcJbiiomJgZ2dHaKjo1GKG7wQERFlWXauoYp1xyQkJODUqVNo06ZNSjAWFmjTpg2OHj1q9DVNmzbFqVOn9F02169fx+7du9G+fXuDOqGhobh8+TIA4OzZszh8+DDatWuXbizx8fGIiYkxOIiIiCh/KdYd8+jRI2i1Wtjb2xuU29vb49KlS0Zf07dvXzx69Aivv/46hBBISkrCxx9/jIkTJ+rr+Pv7IyYmBjVr1oRarYZWq8Xs2bPRr1+/dGMJCAjAl19+mTcfjIiIiLJE8YGp2XHgwAHMmTMH3333HU6fPo2QkBDs2rULM2fO1NfZsmUL1q9fjw0bNuD06dNYu3Ytvv76a6xduzbd806YMAHR0dH6IyIioiA+DhERUZGmWEtI+fLloVarERUVZVAeFRUFBwcHo6+ZMmUK3n//fXz44YcAAC8vL8TGxmLo0KGYNGkSLCwsMH78ePj7+6N37976Ordu3UJAQAAGDBhg9LxWVlawsrLKw09HREREmVGsJcTS0hINGzY0GGSq0+kQGhqKJk2aGH3NixcvYGFhGLL6/1f7Sh5fm14dnU6Xl+ETERFRLik6RXfs2LEYMGAAfHx80LhxYwQFBSE2NhaDBg0CAPTv3x/Ozs4ICAgAAHTs2BGBgYGoX78+fH19cfXqVUyZMgUdO3bUJyMdO3bE7NmzUalSJXh6euLMmTMIDAzE4MGDsxxXckLDAapERETZk3ztzNLkW6GwRYsWiUqVKglLS0vRuHFjcezYMf1zLVu2FAMGDNA/TkxMFNOnTxceHh7C2tpauLq6imHDhoknT57o68TExIhRo0aJSpUqCWtra1GlShUxadIkER8fn+WYIiIiBAAePHjw4MGDRw6PiIiITK+3iq4TYqp0Oh3u3r0LW1tbqNLbga4Qi4mJgaurKyIiIrhOSh7g95n3+J3mLX6fea8of6dCCDx79gxOTk5phkekpmh3jKmysLCAi4uL0mEorlSpUkXuP09+4veZ9/id5i1+n3mvqH6ndnZ2WapnVlN0iYiIqPBgEkJERESKYBJCaVhZWWHatGlcOyWP8PvMe/xO8xa/z7zH7zRrODCViIiIFMGWECIiIlIEkxAiIiJSBJMQIiIiUgSTECIiIlIEkxACAAQEBKBRo0awtbVFxYoV0blzZ4SHhysdVqExd+5cqFQqjB49WulQzFpkZCTee+89lCtXDhqNBl5eXvjnn3+UDstsabVaTJkyBe7u7tBoNPDw8MDMmTOztucH4a+//kLHjh3h5OQElUqFHTt2GDwvhMDUqVPh6OgIjUaDNm3a4MqVK8oEa6KYhBAA4ODBgxg+fDiOHTuGvXv3IjExEW3btkVsbKzSoZm9kydPYvny5ahbt67SoZi1J0+eoFmzZihevDh+++03XLhwAd988w3KlCmjdGhma968eVi6dCkWL16MixcvYt68eZg/fz4WLVqkdGhmITY2FvXq1cOSJUuMPj9//nwsXLgQy5Ytw/Hjx1GyZEn4+fkhLi6ugCM1XZyiS0Y9fPgQFStWxMGDB9GiRQulwzFbz58/R4MGDfDdd99h1qxZ8Pb2RlBQkNJhmSV/f38cOXIEhw4dUjqUQqNDhw6wt7fHqlWr9GXdunWDRqNBcHCwgpGZH5VKhe3bt6Nz584AZCuIk5MTPvvsM4wbNw4AEB0dDXt7e6xZswa9e/dWMFrTwZYQMio6OhoAULZsWYUjMW/Dhw/HO++8gzZt2igditn7+eef4ePjgx49eqBixYqoX78+Vq5cqXRYZq1p06YIDQ3F5cuXAQBnz57F4cOH0a5dO4UjM383btzA/fv3Df7v29nZwdfXF0ePHlUwMtPCDewoDZ1Oh9GjR6NZs2aoU6eO0uGYrU2bNuH06dM4efKk0qEUCtevX8fSpUsxduxYTJw4ESdPnsSnn34KS0tLDBgwQOnwzJK/vz9iYmJQs2ZNqNVqaLVazJ49G/369VM6NLN3//59AIC9vb1Bub29vf45YhJCRgwfPhznz5/H4cOHlQ7FbEVERGDUqFHYu3cvrK2tlQ6nUNDpdPDx8cGcOXMAAPXr18f58+exbNkyJiE5tGXLFqxfvx4bNmyAp6cnwsLCMHr0aDg5OfE7pQLB7hgyMGLECPz666/Yv38/XFxclA7HbJ06dQoPHjxAgwYNUKxYMRQrVgwHDx7EwoULUaxYMWi1WqVDNDuOjo6oXbu2QVmtWrVw+/ZthSIyf+PHj4e/vz969+4NLy8vvP/++xgzZgwCAgKUDs3sOTg4AACioqIMyqOiovTPEZMQ+n9CCIwYMQLbt2/Hn3/+CXd3d6VDMmutW7fGuXPnEBYWpj98fHzQr18/hIWFQa1WKx2i2WnWrFmaaeOXL19G5cqVFYrI/L148QIWFoaXAbVaDZ1Op1BEhYe7uzscHBwQGhqqL4uJicHx48fRpEkTBSMzLeyOIQCyC2bDhg3YuXMnbG1t9X2WdnZ20Gg0CkdnfmxtbdOMpylZsiTKlSvHcTY5NGbMGDRt2hRz5sxBz549ceLECaxYsQIrVqxQOjSz1bFjR8yePRuVKlWCp6cnzpw5g8DAQAwePFjp0MzC8+fPcfXqVf3jGzduICwsDGXLlkWlSpUwevRozJo1C9WqVYO7uzumTJkCJycn/QwaAiCIhBAAjB6rV69WOrRCo2XLlmLUqFFKh2HWfvnlF1GnTh1hZWUlatasKVasWKF0SGYtJiZGjBo1SlSqVElYW1uLKlWqiEmTJon4+HilQzML+/fvN/p7c8CAAUIIIXQ6nZgyZYqwt7cXVlZWonXr1iI8PFzZoE0M1wkhIiIiRXBMCBERESmCSQgREREpgkkIERERKYJJCBERESmCSQgREREpgkkIERERKYJJCBERESmCSQgREREpgkkIERUJBw4cgEqlwtOnT5UOhYj+H5MQIiIiUgSTECIiIlIEkxAiKhA6nQ4BAQFwd3eHRqNBvXr1sHXrVgApXSW7du1C3bp1YW1tjddeew3nz583OMe2bdvg6ekJKysruLm54ZtvvjF4Pj4+Hl988QVcXV1hZWWFqlWrYtWqVQZ1Tp06BR8fH5QoUQJNmzZFeHh4/n5wIkoXkxAiKhABAQH48ccfsWzZMvz7778YM2YM3nvvPRw8eFBfZ/z48fjmm29w8uRJVKhQAR07dkRiYiIAmTz07NkTvXv3xrlz5zB9+nRMmTIFa9as0b++f//+2LhxIxYuXIiLFy9i+fLlsLGxMYhj0qRJ+Oabb/DPP/+gWLFi3LaeSElKb+NLRIVfXFycKFGihPj7778Nyj/44APRp08f/ZbomzZt0j/3+PFjodFoxObNm4UQQvTt21e89dZbBq8fP368qF27thBCiPDwcAFA7N2712gMye+xb98+fdmuXbsEAPHy5cs8+ZxElD1sCSGifHf16lW8ePECb731FmxsbPTHjz/+iGvXrunrNWnSRH+/bNmyqFGjBi5evAgAuHjxIpo1a2Zw3mbNmuHKlSvQarUICwuDWq1Gy5YtM4ylbt26+vuOjo4AgAcPHuT6MxJR9hVTOgAiKvyeP38OANi1axecnZ0NnrOysjJIRHJKo9FkqV7x4sX191UqFQA5XoWICh5bQogo39WuXRtWVla4ffs2qlatanC4urrq6x07dkx//8mTJ7h8+TJq1aoFAKhVqxaOHDlicN4jR46gevXqUKvV8PLygk6nMxhjQkSmjS0hRJTvbG1tMW7cOIwZMwY6nQ6vv/46oqOjceTIEZQqVQqVK1cGAMyYMQPlypWDvb09Jk2ahPLly6Nz584AgM8++wyNGjXCzJkz0atXLxw9ehSLFy/Gd999BwBwc3PDgAEDMHjwYCxcuBD16tXDrVu38ODBA/Ts2VOpj05EGWASQkQFYubMmahQoQICAgJw/fp1lC5dGg0aNMDEiRP13SFz587FqFGjcOXKFXh7e+OXX36BpaUlAKBBgwbYsmULpk6dipkzZ8LR0REzZszAwIED9e+xdOlSTJw4EcOGDcPjx49RqVIlTJw4UYmPS0RZoBJCCKWDIKKi7cCBA2jVqhWePHmC0qVLKx0OERUQjgkhIiIiRTAJISIiIkWwO4aIiIgUwZYQIiIiUgSTECIiIlIEkxAiIiJSBJMQIiIiUgSTECIiIlIEkxAiIiJSBJMQIiIiUgSTECIiIlLE/wFs8uxom/g2KgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* background: */\n",
       "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
       "    progress {background-color: #CDCDCD;}\n",
       "\n",
       "    /* value: */\n",
       "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
       "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
       "    progress {color: #00BFFF ;}\n",
       "\n",
       "    /* optional */\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #000000;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      <progress value='11' class='progress-bar-interrupted' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      11.00% [11/100 04:26<35:54][earlystopping]\n",
       "      <br>\n",
       "      \n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< val_acc without improvement in 10 epoch,early stopping >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>train_acc</th>\n",
       "      <th>lr</th>\n",
       "      <th>val_loss</th>\n",
       "      <th>val_acc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.306164</td>\n",
       "      <td>0.883464</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.224274</td>\n",
       "      <td>0.922842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.225757</td>\n",
       "      <td>0.914714</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.221378</td>\n",
       "      <td>0.912946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.181335</td>\n",
       "      <td>0.936589</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.249262</td>\n",
       "      <td>0.914583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.150235</td>\n",
       "      <td>0.949219</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.265122</td>\n",
       "      <td>0.907292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.122353</td>\n",
       "      <td>0.958073</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.300408</td>\n",
       "      <td>0.911458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0.096260</td>\n",
       "      <td>0.967708</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.287057</td>\n",
       "      <td>0.900521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0.081176</td>\n",
       "      <td>0.973698</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.356954</td>\n",
       "      <td>0.901563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0.065423</td>\n",
       "      <td>0.978739</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.354140</td>\n",
       "      <td>0.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>0.051379</td>\n",
       "      <td>0.981771</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.397223</td>\n",
       "      <td>0.902604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>0.046847</td>\n",
       "      <td>0.984115</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.412438</td>\n",
       "      <td>0.901042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>0.052747</td>\n",
       "      <td>0.982422</td>\n",
       "      <td>0.00003</td>\n",
       "      <td>0.360155</td>\n",
       "      <td>0.897396</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    epoch  train_loss  train_acc       lr  val_loss   val_acc\n",
       "0       1    0.306164   0.883464  0.00003  0.224274  0.922842\n",
       "1       2    0.225757   0.914714  0.00003  0.221378  0.912946\n",
       "2       3    0.181335   0.936589  0.00003  0.249262  0.914583\n",
       "3       4    0.150235   0.949219  0.00003  0.265122  0.907292\n",
       "4       5    0.122353   0.958073  0.00003  0.300408  0.911458\n",
       "5       6    0.096260   0.967708  0.00003  0.287057  0.900521\n",
       "6       7    0.081176   0.973698  0.00003  0.356954  0.901563\n",
       "7       8    0.065423   0.978739  0.00003  0.354140  0.900000\n",
       "8       9    0.051379   0.981771  0.00003  0.397223  0.902604\n",
       "9      10    0.046847   0.984115  0.00003  0.412438  0.901042\n",
       "10     11    0.052747   0.982422  0.00003  0.360155  0.897396"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_model.fit(\n",
    "    train_data = dl_train,\n",
    "    val_data= dl_val,\n",
    "    ckpt_path='bert_waimai.pt',\n",
    "    epochs=100,\n",
    "    patience=10,\n",
    "    monitor=\"val_acc\", \n",
    "    mode=\"max\",\n",
    "    plot = True,\n",
    "    wandb = False,\n",
    "    quiet = True\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四，评估模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以使用huggingFace的evaluate库来进行模型评估。\n",
    "\n",
    "通过evaluate的load方法可以加载一些常用的评估指标。\n",
    "\n",
    "可以用add_batch逐批次地往这些评估指标上添加数据，最后用compute计算评估结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install evaluate "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'accuracy': 0.9128440366972477}"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import evaluate\n",
    "metric = evaluate.load(\"accuracy\")\n",
    "model.eval()\n",
    "dl_test = keras_model.accelerator.prepare(dl_test)\n",
    "for batch in dl_test:\n",
    "    with torch.no_grad():\n",
    "        outputs = model(**batch)\n",
    "\n",
    "    logits = outputs.logits\n",
    "    predictions = torch.argmax(logits, dim=-1)\n",
    "    metric.add_batch(predictions=predictions, references=batch[\"labels\"])\n",
    "\n",
    "metric.compute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 五，使用模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "texts = [\"味道还不错，下次再来\",\"这他妈也太难吃了吧\",\"感觉不是很新鲜\",\"还行我家狗狗很爱吃\"]\n",
    "batch = tokenizer(texts,padding=True,return_tensors=\"pt\")\n",
    "batch = {k:v.to(keras_model.accelerator.device) for k,v in batch.items()}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.9510, 0.0133, 0.1020, 0.6223], device='cuda:0',\n",
      "       grad_fn=<SelectBackward0>)\n"
     ]
    }
   ],
   "source": [
    "from torch import nn \n",
    "logits = model(**batch).logits \n",
    "scores = nn.Softmax(dim=-1)(logits)[:,-1]\n",
    "print(scores)\n",
    "#可以看到得分与人的预期是高度一致的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也可以用pipeline把tokenizer和model组装在一起"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Xformers is not installed correctly. If you want to use memorry_efficient_attention to accelerate training use the following command to install Xformers\n",
      "pip install xformers.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'label': 'LABEL_1', 'score': 0.9468138813972473}]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import pipeline\n",
    "classifier = pipeline(task=\"text-classification\",tokenizer = tokenizer,model=model.cpu())\n",
    "classifier(\"挺好吃的哦\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 六，保存模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "保存model和tokenizer之后，我们可以用一个pipeline加载，并进行批量预测。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('waimai_10k_bert/tokenizer_config.json',\n",
       " 'waimai_10k_bert/special_tokens_map.json',\n",
       " 'waimai_10k_bert/vocab.txt',\n",
       " 'waimai_10k_bert/added_tokens.json',\n",
       " 'waimai_10k_bert/tokenizer.json')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.config.id2label = {0:\"差评\",1:\"好评\"}\n",
    "model.save_pretrained(\"waimai_10k_bert\")\n",
    "tokenizer.save_pretrained(\"waimai_10k_bert\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import pipeline \n",
    "classifier = pipeline(\"text-classification\",model=\"waimai_10k_bert\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': '好评', 'score': 0.950958251953125},\n",
       " {'label': '差评', 'score': 0.9617311954498291}]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier([\"味道还不错，下次再来\",\"我去，吃了我吐了三天\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**如果本项目对你有所帮助，想鼓励一下作者，记得给本项目加一颗星星star⭐️，并分享给你的朋友们喔😊!** \n",
    "\n",
    "如果在torchkeras的使用中遇到问题，可以在项目中提交issue。\n",
    "\n",
    "如果想要获得更快的反馈或者与其他torchkeras用户小伙伴进行交流，\n",
    "\n",
    "可以在公众号算法美食屋后台回复关键字：**加群**。\n",
    "\n",
    "![](https://tva1.sinaimg.cn/large/e6c9d24egy1h41m2zugguj20k00b9q46.jpg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
